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REDUCING BLOCKING AND RINGING ARTIFACTS EST LOW-BIT-RATE 
CODING 

Inventor: Changick Kim 



5 BACKGROUND OF THE INVENTION 

Field of the Invention 

This present invention relates to an improved algorithm for smoothing 
artificial discontinuities between image blocks (blocking artifacts) and reducing 
ringing noise, without introducing undesired blur. The invention, which can be 
10 embodied in an apparatus, methods or programs of instructions, is particularly, 

□ though not exclusively, designed for low-bit-rate video coding applications. 

O 

s i Description of the Related Art 

m Today's low-bit-rate video coding standards, MPEG-4 and ITU-T H.263, 

" 4 contain algorithms that enable a variety of applications, including video 

145 conferencing and wireless video communications. These standards and the systems 

ffj that use them take advantage of temporal redundancy as well as spatial 

ti redundancy to compress video. While these standards are quite effective in many 

H ways, they sometimes generate decompressed images that exhibit artificial 
discontinuities between image blocks. These blocking artifacts are caused primarily 

20 by quantization during the quantization step of the compression process. 

To combat this problem these coding standards have incorporated algorithms 
for reducing blocking artifacts. In block-based coding, monotone areas of the 
original image, where the pixel intensity changes gradually, suffer most noticeably 
from the abrupt changes across the block boundary, leading to blocking artifacts. In 

25 terms of discrete cosine transform (DCT), when the DCT coefficient quantization 
step size is above the threshold for visibility, discontinuities in grayscale values are 
caused by removal of AC coefficients due to quantization and become clearly visible 
at the boundaries between blocks. Also, visually annoying noise near real edges, 
which is called ringing noise, is inevitable in the block-based DCT scheme. In 

30 applications using low bit-rate coded video sources, such blocking and ringing 
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artifacts cause substantial practical problems, and therefore must be reduced. 
Various deblocking schemes have been proposed in still image coding such as JPEG 
under the assumption that blocking artifacts are always located at block 
boundaries. A well-known method for reducing blocking artifacts is based on the 
5 theory of alternative projection onto convex sets (POCS). However, this method is 
only applied to still images because of an iteration structure and long convergence 
time. 

In video coding, unlike still image coding, blocking artifacts might occur 
inside a block since the blocking artifacts of the previous frame can be propagated 
10 to the current frame. Recently, post-filtering methods for low bit-rate coded video 
have been proposed. For example, an adaptive filtering scheme has been proposed 

□ based on local characteristics to reduce blocking artifacts. In another proposal a 

□ 

y frequency-domain analysis is conducted for extraction of the block semaphores from 
the intracoded picture. The semaphores extracted are propagated using inverse 

1 1 5 motion compensation for inter-frame de-blocking and de-ringing. While these 
techniques are generally acceptable, further improvements, especially in the areas 
of increased speed and reduced computational complexity, are desirable. 

OBJECTS AND SUMMARY OF THE INVENTION 

Objects of the Invention 

20 A main object of the present invention is to provide an improved technique, ' 

having low computational complexity and high speed, for reducing blocking and 
ringing artifacts from still images or video frames based on local characteristics. 

It is another object of this invention to provide an improved technique for 
reducing blocking and ringing artifacts which employs block-based adaptive 
25 filtering based on block classification and certain characteristics of the 
corresponding predicted transform coefficients. 

Summary of the Invention 

According to one aspect of this invention, a method for reducing artifacts in 
digital data is provided. Such method comprises the steps of: (a) obtaining a 
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reconstructed block-based pixel representation of the digital data; (b) extracting a 
DC coefficient for each block in the pixel representation based on values of selected 
pixels in that block to generate a map of DC coefficients; (c) for each pixel block, 
predicting a select number of lower frequency AC coefficients using the DC 
5 coefficient for that pixel block and a select number of neighboring DC coefficients in 
the DC coefficient map to construct a corresponding partial transform-coefficient 
block; (d) classifying each pixel block based on the predicted AC coefficients in the 
corresponding partial transform-coefficient block; and (e) selectively performing a 
low -pass filtering operation on select pixels in select pixel blocks on a block-by -block 
10 basis based on the classification of that block, the locations and values of predicted 
non-zero AC coefficients in the corresponding partial transform block, and certain 

pi pixel values in that block. 

C3 

The following additional features may also be included in the above-described 
method. 

\15 In step (c), the five lowest frequency AC coefficients are predicted for each 

pixel block. 

y In step (d), the classifying of each pixel block comprises comparing the 

absolute sum of the predicted AC coefficients in the corresponding partial transform 
M block to a threshold. 

20 In step (d), a particular pixel block is classified as a low-activity block if the 

absolute sum of the predicted AC coefficients in the corresponding partial 
transform-coefficient block is less than the threshold, and, if not, the particular 
pixel block is classified as a high-activity block. 

Step (e) comprises applying a strong low-pass filter to boundary region pixels 
25 in select low-activity blocks and applying a weak low-pass filter to pixels in select 
high-activity blocks. 

For each low-activity block, the strong low-pass filter is applied to: smooth 
boundary region pixels in both the horizontal and vertical directions if all of the 
predicted AC coefficients in the corresponding partial transform-coefficient block 
30 have absolute values less than a predetermined value; smooth boundary region 
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pixels in the vertical direction if at least one AC coefficient in a first select location 
in the corresponding partial transform-coefficient block has an absolute value 
greater than or equal to the predetermined value; and smooth boundary region 
pixels in the horizontal direction if at least one AC coefficient in a second location in 
5 the corresponding partial transform-coefficient block has an absolute value greater 
than or equal to the predetermined value. 

For each low-activity block, the strong low-pass filter is applied to a current 
boundary region pixel only when a difference between a maximum pixel value and a 
minimum pixel value among a predetermined number of pixels centered on the 
10 current boundary region pixel is less than a predetermined parameter. This 
y prevents real edges which may exist in a low-activity block due to an incorrect 
Sf classification decision from being blurred by application of a strong low-pass filter. 

Zji For each high-activity block, the weak low-pass filter is applied to smooth 
pixels inside the block in the horizontal direction if a difference between values of a 

15 first two adjacent pixels and a difference between values of a second two adjacent 

§=* pixels are both less than a predetermined parameter and is applied to smooth pixels 

: : inside the block in the vertical direction if a difference between values of a third two 

: : adjacent pixels and a difference between values of a fourth two adjacent pixels are 

jU both less than the predetermined parameter. 

20 In another aspect, the invention involves an apparatus for reducing artifacts 

in digital data. Such apparatus comprises: a DC-coefficient-map generator that 
receives a reconstructed block-based pixel representation of the digital data and 
extracts a DC coefficient for each block in the pixel representation based on values 
of selected pixels in that block to generate a map of DC coefficients; an AC 

25 coefficient predictor that, for each pixel block, predicts a select number of lower 
frequency AC coefficients using the extracted DC coefficient for that pixel block and 
a select number of neighboring DC coefficients in the DC coefficient map to 
construct a corresponding partial transform-coefficient block; a block classifier that 
classifies each pixel block based on the predicted AC coefficients in the 

30 corresponding partial transform-coefficient block; and an adaptive filtering unit 
that selectively performs a low-pass filtering operation on select pixels in select 
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pixel blocks on a block-by -block basis based on the classification of that block, the 
locations and values of predicted non-zero AC coefficients in the corresponding 
partial transform block, and certain pixel values in that block. 

Preferably, the block classifier classifies a particular pixel block as a low- 
5 activity block if the absolute sum of the predicted AC coefficients in the 
corresponding partial transform-coefficient block is less than the threshold, and, if 
not, classifies the particular pixel block as a high-activity block. 

Preferably, the adaptive filtering unit comprises a strong low-pass filter that 
is applied to boundary region pixels in select low-activity blocks, and a weak low- 
10 pass filter that is applied to pixels in select high-activity blocks. 

In accordance with further aspects of the invention, any of the above- 
described methods or steps thereof may be embodied in a program of instructions 
s 4 (e.g., software) which may be stored on, or conveyed to, a computer or other 
fj4 processor-controlled device for execution. Alternatively, any of the methods or steps 
15 thereof may be implemented using functionally equivalent hardware (e.g., 
7: application specific integrated circuit (ASIC), digital signal processing circuitry, 
ry etc.) or a combination of software and hardware. 

Other objects and attainments together with a fuller understanding of the 
invention will become apparent and appreciated by referring to the following 
20 description and claims taken in conjunction with the accompanying drawings. 

BRIEF DESCRIPTION OF THE DRAWINGS 

Fig. 1 is a block diagram illustrating the processing pipeline in accordance 
with embodiments of the present invention. 

Figs. 2(a), 2(b) and 2(c) are schematic diagrams illustrating DC coefficients 
25 extraction, DC coefficient map generation and AC coefficient prediction. 

Figs. 3(a) and 3(b) are schematic diagrams illustrating the pixel locations to 
which strong low-pass niters are applied in low-activity blocks. 

Fig. 4 is an image with recognized high-activity blocks marked. 
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Fig. 5 is a schematic diagram illustrating the pixel locations to which weak 
low-pass filters are applied in high-activity blocks. 

Fig. 6 is a flow chart illustrating certain process steps of the algorithm in 
accordance with embodiments of the invention. 

5 Fig. 7 is a block diagram illustrating components in an exemplary image 

processing system that may be used to implement aspects of the present invention. 

DESCRIPTION OF THE PREFERRED EMBODIMENTS 
A. Overview 

M The present invention provides a technique (e.g., an algorithm) to reduce 

10 blocking and ringing artifacts in low bit-rate block-based coding, and is particularly 
applicable to such video coding. Generally, for each reconstructed frame block (i.e., 
2f a pixel block), the algorithm estimates its DC frequency coefficient and predicts 
SJ certain AC frequency coefficients, classifies each block based on the predicted AC 
jL fe coefficients, and adaptively filters select block regions based on the block 
:15 classification to reduce blocking and ringing artifacts. As illustrated in Fig. 1, the 
W technique is applied to each reconstructed frame output from a decoder 10. 
P 

First a map of the DC frequency coefficients is generated (block 11). The DC 
value of each reconstructed block in the map is determined by sampling the pixel 
values at 8 out of 64 locations in the block. From this DC coefficient map, each DC 
20 value and the eight surrounding DC values in the map are exploited to predict the 
five lowest frequency AC coefficients (block 12) which might be lost in the 
quantization stage of the encoding process. 

It should be noted that the sampling of 12.5% of the pixels in a block (e.g., 8 
out of 64 pixels in a JPEG or MPEG scheme) is a preferred way of determining the 

25 block's DC coefficient; however, it is not the only way. Other reasonable variations 
of sub-sampling may be employed, such as sampling anywhere between 10% and 
20%, recognizing the trade off between accuracy and computational complexity. 
Likewise, other reasonable variations may be employed in terms of the number of 
AC coefficients predicted and the DC coefficients used to make the prediction. For 

30 example, instead of using all eight surrounding DC coefficients in the DC coefficient 
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map, a lesser number may be used. Also, the number of AC coefficients may be 
slightly more or less than five. Again, the trade off is one of accuracy vs. 
computational complexity. Such variations will be apparent to one skilled in the art 
in light of the disclosure herein. 

5 The predicted AC coefficients are used to classify each reconstructed block as 

either a low- or high-activity block (block 13). A "low-activity block" is a block 
(typically 8x8) wherein the spatial standard deviation is small. A "high-activity 
block" is a block (typically 8x8) wherein there is at least one real edge; thus the 
spatial standard deviation is large. 

10 Next, two kinds of low-pass filters (LPF) are adaptively applied, on a block- 

M by-block basis, according to the classification of the particular block (block 14). A 
O strong LPF is applied to boundary region pixels in low-activity blocks where the 
blocking artifacts are most noticeable, whereas a weak LPF is applied to pixels in 
y high-activity blocks. The adaptive filtering is designed to reduce ringing noise as 
"--15 well as blocking artifacts without introducing undesired blur. In low-activity-block 
jj\ filtering, the blocking artifacts are reduced by one dimensional horizontal and 
|=j vertical LPFs, which are selectively applied in the horizontal and/or vertical 
yj direction depending on the locations and absolute values of the predicted AC 
lI coefficients. In high-activity-block filtering, de-blocking and de-ringing is achieved 
20 by a single filter, applied horizontally and/or vertically, which makes the invented 
architecture simple. 



B. Algorithm 

Blocking artifacts occurring in still image coding such as JPEG originate 
from the quantization of DCT coefficients in the encoding process. In the JPEG 

25 standard, a technique for predicting the AC coefficients is recommended in Annex- 
K.8.2 as an option for the decoder in order to suppress between-block discontinuities 
of the decoded image. In this approach, the image is assumed to be a quadratic 
surface, and the missing low frequency coefficients are predicted to fit this surface. 
However, in areas with sharp intensity transitions, this model may fail to provide 

30 accurate AC coefficients. Nevertheless, the inventor of this invention has observed 
that this AC prediction scheme provides useful information that may be used to 
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classify blocks and apply low-pass filters adaptively according to the predicted AC 
coefficient values. 

Accordingly, the present invention classifies blocks and allows for adaptive 
selection of the strength and direction of filtering. Thus, unlike previous methods 
5 that require edge detection, the algorithm of this invention does not need to 
compute edge information and apply an LPF adaptively based on the edges. The 
present invention comprises three primary functional blocks, i.e., AC coefficients 
prediction, block classification, and adaptive filtering depending on block 
classification results. Each of these functional blocks is described below in 
10 connection with a preferred embodiment of the invention. As previously noted, 
Li ; other variations are possible. 

yj B.l AC Coefficients Prediction and Block Classification 

,J Initially, DC coefficient values are extracted from the reconstructed frame 

= : blocks to generate a map of DC coefficients. To reduce computational load, the DC 
;15 value of each block is calculated by sampling the pixel values at 8 out of 64 
□ locations in that block. The generated DC coefficient map has a reduced size by a 
factor of 8 (i.e., 22 x 18 in case of QCIF), requiring little additional memory. (QCIF 
stands for Quarter Common Intermediate Format, which is a video format defined 
in ITU-T Recommendation H.261 and characterized by 176 luminance pixels on 
20 each of 144 lines, with half as many chrominance pixels in each direction.) In the 
illustrated embodiment, each DC value is 8 times the approximation of the mean 
value of its corresponding reconstructed block. 

The process of generating a map of DC coefficients is schematically 
illustrated in Figs. 2(a) and 2(b). Fig. 2(a) shows a 3 x 3 array of reconstructed 

25 blocks, one of which (e.g., the middle one) is further divided into an 8 x 8 array of 
pixels. The DC value for each block is extracted by sampling the values of the 8 
darkened pixels in that block to produce the corresponding value in the 3x3 DC 
coefficient map shown in Fig. 2(b). The sampling of the middle reconstructed block 
produces DC S in the DC coefficient map. The other DC coefficients are produced by 

30 sampling pixel values in their respective reconstructed blocks. 
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After the DC coefficient map is generated from the reconstructed frame, the 
DC value of the current block and its surrounding eight neighbor DC values are 
exploited to predict the five lowest-frequency AC coefficients of the corresponding 
DCT block, as schematically illustrated in Fig. 2(c), and more fully explained below. 

A quadratic surface, given by 

P(x,y) = A } x 2 y 2 +A 2 x 2 y + A i xy 2 +A 4 x 2 +A 5 xy + A 6 y 2 +A 1 x + A ji y + A 9 (1) 

is fitted to the 3x3 array of DC values, and is used to estimate the five lowest- 
frequency AC coefficients of the 8x8 pixel array in the central block. The 
coefficients A l ...A 9 are determined by requiring that the mean values computed for 
the quadratic surface match the DC values, £>C, ...DC 9 , with appropriate scaling for 
the DCT normalization. 

If a DCT is computed for the central block, the result is a set of equations 
relating the AC coefficients required to reproduce the quadratic surface to the DC 
coefficients which predict the quadratic surface. In one embodiment, the equations 
for the first five coefficients in the zigzag scan are: 

A C 01 = (1 . 1 3 885 / 8)(£>C 4 - DC 6 ) 
AC l0 = (1.1 3885/ 8)(Z>C 2 -DC S ) 

AC 20 = (0.2788 1 / 8)(DC 2 +DC S -2DC 5 ) (2) 
A Q j = (0. 1 62 1 3 / 8)((DC, - DC 3 ) - (DC 7 - DC 9 )) 
AC Q2 = (0.2788 1/8)(Z)C 4 +DC 6 -2DC 5 ) . 

The subscripts of the AC coefficients indicate the horizontal and vertical 
positions of the coefficient in the DCT array (see Fig. 2(c)). These equations predict 
the unquantized AC coefficients from the unquantized DC values. The original 
block activity can be expressed by the absolute sum of the predicted AC coefficients, 
since the sum is approximated by the standard deviation of the block which can be 
described as follows. 
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Consider an image f which has IT blocks, each of size NxN . A pixel value at 
0 < i,j < N-l, in the kth block, l<k<K, is denoted by f k (i,j) where the blocks 
are numbered sequentially from 1 beginning at the top left to the bottom right in a 
raster-scan order. The kth block of the image, f k (i,j),0< i,j < N-l, is transformed to 
5 F k (u,v), 0<u,v<N-l, where and (u,v) are coordinate indexes, respectively, in 
the spatial and transform domains. 

In the spatial domain, the mean variance can be used to determine the 
characteristics of the original kth block, where the mean variance is expressed by 
°k =y ^ i ^ y ^ j ~lifk{Uj)-^kf an( i is the mean of fk . From Parseval's energy 
10 theorem, the mean variance can be exactly calculated from the sum of square of the 
| ac components, i.e., a\ = ^fe^X£fc >' v > 2 "^(°,0) 2 ). Define F k (u,v)is a 

predicted coefficient of F k (u, v)by unknown prediction method. If F k (u,v) is 
3 obtainable from the decoding stage, the estimated mean variance of the original 

: block, 6 2 k =^{^~qY!^o^(^ v ) 2 -^(^) 2 ) can be calculated. To reduce 

£3 

i5 computation, the estimated standard deviation is approximated by the mean 

2; absolute values as 

Ignoring high frequency coefficients except the five lowest AC coefficients 
predicted from Eq. (2), the standard deviation expressed in Eq. (3) is approximated 
20 as 

= jj H (0,1)| + \F k (1,0)| + \F k (2,0)| + \F k (1,1)| + \F k (0,2)|) . (4) 

Thus, by replacing notations and ignoring scaling factor, the block activity of the 
kth block is defined as 

ACT, = \AC Q1 \ + \AC l0 \+ \AC 20 \+ \AC U \+ \AC 02 \ . (5) 
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Since the standard deviation in each of the low-activity blocks has a small 
absolute value, the equation below is used to classify the block characteristics. 

Uf(\A C 01 1 + \AC l0 1 + \AC 20 1 + \AC U | + \AC 02 \ < 6), low - activity block (g) 
[otherwise, high - activity block 

If the absolute sum of the predicted AC coefficients is smaller than a preset 
threshold 0 , the corresponding reconstructed block is deemed to be a low-activity 
block. Otherwise, that reconstructed block is considered to be a high-activity block. 
The threshold value may be determined by simulations for various video sequences. 
In one embodiment, the threshold 0 may be set to, for example, 400. 

B.2 Adaptive Filtering using Classification Results 

In this stage, adaptive low-pass filtering is performed on certain 
reconstructed blocks, on a block-by-block basis, depending on block classification, 
the locations of the predicted non-zero AC coefficients, and certain pixel values, as 
described below. 

For the low-activity blocks, a one-dimensional (1-D) strong (e.g., 7-tab) LPF is 
applied to boundary region pixels, along all of the horizontal and vertical edges. 
The predicted AC coefficients can also be exploited for selective application of 
horizontal and/or vertical strong LPFs depending on the locations and absolute 
values of the predicted AC coefficients. Such coefficients reflect spatial activity of 
the original block and are assumed to be the unquantized coefficients of the intra- 
coded blocks. Since the quantization formula used for intra-coded blocks is: 




(-\,coeff<Q\ 
Q,coeff = o\ and QPis a quantization parameter that is determined 
+ \,coeff>0\ 

in the encoding stage by the user, the blocks which have predicted coefficients 
whose absolute values are less than 2xQP are expected to have blocking artifacts 
in the decoded frame. 
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In one embodiment, for each low-activity block, three situations are 
considered for strong low-pass filtering. When all of the predicted AC coefficients 
have absolute values less than 2xQP, it is assumed that the 64 pixels of the 8x8 
decoded block have the same values in the spatial domain; therefore, such a block is 
5 a candidate for having a strong LPF applied to it in both the horizontal and vertical 
directions. When only the predicted coefficients for position AC 01 and/or position 
v4C 02 of the 8x8 inverse quantized block have absolute values greater than or equal 
to 2 xQP, it is expected that the eight pixels in each column have the same value in 
the spatial domain, while there exists fluctuation along each row. This block may 
10 induce blocking artifacts at the top and the bottom of the block, and hence is a 
candidate for having a strong LPF applied to it in the vertical direction. When only 
O the predicted coefficients for position AC W and/or position AC 2Q have absolute 
yJ values greater than or equal to 2 x QP , it is expected that the eight pixels in each 
%f row have the same values in the spatial domain, while there exists fluctuation 
15 along each column. This block may induce blocking artifacts at the left and the 
* . right of the block, and hence is a candidate for having a strong LPF applied to it in 
j==j; the horizontal direction. 

The coefficients of such a strong LPF are preferably as follows: 

P 1 

fc(«) = -i{Ll,l,2,U,l} (7) 

20 and filtering is performed, subject to the below-described condition, on boundary 
region pixels, that is, three rows/columns of pixels on either side of the block 
boundary. Figs. 3(a) and 3(b) show the pixel locations to which the horizontal and 
vertical LPFs are respectively applied in low-activity blocks. Sometimes a block 
containing real edges may be classified as a low-activity block, which may occur 

25 when the DC value of the block is very similar to those of neighbor blocks. Thus, to 
prevent real edges in the low-activity-classified block from being smoothed, the 
above-described low-activity-block filtering scheme is subject to the following 
condition. Specifically, filtering is not performed when the difference between the 
maximum pixel value and the minimum pixel value among nine pixels centered on 
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the current pixel is greater than QP . For instance, pixel P 5 shown in Fig. 3(a) is 
not low-pass filtered if max(P) - min(P) > QP, where P = {P l ,P 2 ,...,P s ,P 9 }. 

Unlike low-activity areas, high-activity areas may have visually annoying 
ringing artifacts as well as blocking artifacts. Fig. 4 shows an image with 
5 recognized high-activity blocks marked. The ringing noise occurs as a result of the 
Gibb's phenomenon due to truncation of the high frequency coefficients by 
quantization, which is noticeable near the real edges which correspond to high- 
activity areas. Thus, the filtering in the high-activity areas needs to reduce ringing 
noise as well as blocking artifacts, but without smoothing real edges. A weak (e.g., 
10 3-tab) LPF is chosen for this task. The filter coefficients are preferably as follows: 

| Kn) = j{l, 2,1} (8) 

rj Horizontal and/or vertical filtering is performed on pixels in the high-activity as 

2? long as the difference between adjacent pixels values under the filter is less than 

vj 1 .5 x QP . In other words, if a pixel difference is greater than 1 .5 x QP , it is assumed 

pi> that there is an edge point in three pixels and filtering is not performed. Fig. 5 

7- schematically illustrates the pixel positions to which the horizontal and/or vertical 

^ 3-tab filters are applied. This filtering process can be summarized as follows: 

// ((.41 < 1 .5 * QP) and (A2 < 1 .5 * QP)) , apply horizontal filtering, 

if ((Bl < 1 . 5 * QP) and (B2 < 1 .5 * QP)) , apply vertical filtering, 

20 where 

Al = \pixel[m][n] - pixel[m][n - 1]| 
A2 = \pixel[m][n] - pixel[m][n + 1]| 
Bl = \pixel[m][n]- pixel [m-l][n]\ 
B2 = \pixel [m] [n] - pixel [m + 1] [n]\ 

This filter applied to block boundaries reduces the blocking artifacts and 
reduces ringing noise as well inside the block. This filtering scheme efficiently 
performs de-ringing as well as weak filtering in the high-activity blocks. 
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A flow chart, shown in Fig. 6, summarizes the algorithm in accordance with 
embodiments of the invention. After the algorithm starts, the DC coefficients of the 
reconstructed frame blocks of the digital input representation are extracted to 
generate a DC coefficient map (step 51). Next, in step 52, for each block in the map, 
5 the DC value and its surrounding eight neighbor DC values are used to predict the 
five lowest-frequency AC coefficients of the corresponding DCT block. 

Next, adaptive low-pass filtering is applied to certain pixels of each 
reconstructed frame block on a block-by-block basis using the values of the 
extracted DC coefficient and the five predicted AC coefficients from the 
10 corresponding DCT block. Thus, the following steps of the algorithm are applied to 
each reconstructed frame block until all blocks have been processed. 

For the current DCT block, it is determined in step 53 whether or not the 
^ absolute sum of the predicted AC coefficients is less than a threshold value. If so, 
01 the algorithm proceeds to step 54 where the corresponding reconstructed frame 
15 block is classified as a low-activity block. For the current boundary region pixel, the 
!T maximum pixel value and the minimum pixel value among nine consecutive pixels 
Pi centered on the current pixel are obtained in step 55. If it is determined in step 56 
p that the absolute value of the difference between the maximum and minimum 
values is less than QP , then a 7-tab strong LPF is applied to boundary region pixels 
20 around the horizontal and/or vertical block boundaries in this low-activity block in 
step 57, depending on the location and absolute values of predicted AC coefficients, 
as previously described, after which the algorithm determines whether there is 
another pixel to consider (step 58). If so, the algorithm proceeds through the "yes" 
terminal of decision block 59, obtains the next pixel in step 60 and then returns to 
25 step 55 for the newly obtained pixel. If the decision in step 56 is "no," the algorithm 
proceeds directly to step 58. If it determined at that step that there is not another 
pixel to consider in the current reconstructed block, the algorithm returns to step 53 
for the next DCT block and corresponding reconstructed block until all such blocks 
have been processed, after which the algorithm ends. 

30 Returning to decision step 53, if the absolute sum of the predicted AC 

coefficients is not less than the threshold value, then the algorithm proceeds to step 
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61 where the corresponding reconstructed frame block is classified as a high-activity 
block. 

For the current pixel in the high-activity block, the value differences between 
each pair of adjacent pixels under the 3-tab LPF are determined in step 62. If it is 
5 determined in step 63 that the pixel differences are all less than 1.5 * QP , then a 3- 
tab weak LPF is applied to the current pixel in this high-activity block in step 64, as 
previously described, after which the algorithm determines whether there are any 
more "unprocessed" pixels in the block (step 58). If so, the algorithm proceeds 
through the "no" terminal of decision block 59, obtains the next pixel in step 65 and 
10 then returns to step 62 for the newly obtained pixel. If the decision in step 63 is 
"no," the algorithm proceeds directly to step 58. If it determined at that step that 

O there is not another pixel to consider in the current reconstructed block, the 

o 

algorithm returns to step 53 for the next DCT block and corresponding 
reconstructed block until all such blocks have been processed, after which the 
IB algorithm ends. 

^ C. Effects and Implementations 

fjj As the foregoing description demonstrates, the present invention provides a 

S fast and computationally efficient de-blocking and de-ringing algorithm that is 
fB * particularly applicable to low-bit-rate block-based video coding. The algorithm is 
20 robust to different images and compression schemes. Moreover, because of its 
relatively simple architecture, the algorithm can be implemented in real-time 
applications such mobile video phone systems. The algorithm may be conveniently 
embodied in the form of software that can be used with an existing coding scheme, 
such as MPEG-4 and ITU-T H.263. 

25 Fig. 7 illustrates a typical system which may be used to "run" the de-blocking 

and de-ringing algorithm of the present invention. As illustrated in Fig. 7, system 
70 includes a central processing unit (CPU) 71 that provides computing resources 
and controls the computer. CPU 71 may be implemented with a microprocessor or 
the like, and may also include a graphics processor and/or a floating point 

30 coprocessor for mathematical computations. System 70 further includes system 
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memory 72 which may be in the form of random-access memory (RAM) and read- 
only memory (ROM). 

A number of controllers and peripheral devices are also provided, as shown in 
Fig. 7. Input controller 73 represents an interface to one or more input devices 74, 
5 such as a keyboard, mouse, stylus or remote control. There is also a controller 75 
which communicates with a scanner 76 or equivalent device for digitizing documents 
including images to be processed in accordance with the invention. A storage 
controller 77 interfaces with one or more storage devices 78 each of which includes a 
storage medium such as magnetic tape or disk, or an optical medium that may be 
10 used to record programs of instructions for operating systems, utilities and 
applications which may include embodiments of programs that implement various 
□ aspects of the present invention. Storage device(s) 78 may also be used to store 
data to be processed in accordance with the invention. A display controller 79 
'H provides an interface to a display device 81 which may be a cathode ray tube (CRT), 
ft5 thin film transistor (TFT) display, or video player. A printer controller 82 is also 
provided for communicating with a printer 83 for printing documents including 
images processed in accordance with the invention. A communications controller 84 
ry interfaces with a communication device 85 which enables system 70 to connect to 
? remote devices through any of a variety of networks including the Internet, a local 
? 2t) area network (LAN), a wide area network (WAN), or through any suitable 
electromagnetic carrier signals including infrared signals. 

In the illustrated embodiment, all major system components connect to bus 86 
which may represent more than one physical bus. However, it should be noted that 
while all system components may typically be located in physical proximity to one 

25 another, such is not a requirement of the invention. For example, the input data 
(e.g., reconstructed frame(s)) and/or the output data (e.g., de-blocked/de-ringed 
frame(s)) may be remotely transmitted from one physical location to another. Also, 
programs that implement various aspects of this invention may be accessed from a 
remote location (e.g., a server) over a network. Such data and/or programs may be 

30 conveyed through any of a variety of machine-readable medium including magnetic 
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tape or disk or optical disc, network signals, or other suitable electromagnetic 
carrier signals including infrared signals. 

While the present invention may be conveniently implemented with software, 
a hardware implementation or combined hardware/software implementation is also 
5 possible. A hardware implementation may be realized, for example, using ASIC(s), 
digital signal processing circuitry, or the like. For example, a single ASIC can be 
fabricated that contains the appropriate hardware for performing the functions 
depicted in blocks 11-14 of Fig. 1. The term "apparatus" in the claims is intended to 
cover all such hardware implementations of the invention, as well as other super- 
10 set devices, such as a computer or a system of the type shown in Fig. 7, which 
embody the claimed elements in either hardware or software form. 

- : With these implementation alternatives in mind, it is to be understood that 

the block and flow diagrams show the performance of certain specified functions 
si and relationships thereof The boundaries of these functional blocks have been 
15 denned herein for convenience of description. Alternate boundaries may be defined 
: so long as the specified functions are performed and relationships therebetween are 
£3 appropriately maintained. The diagrams and accompanying description provide the 
y] functional information one skilled in the art would require to write program code 
(i.e., software) or to fabricate circuits (i.e., hardware) to perform the processing 
20 required. 

While the invention has been described in conjunction with several specific 
embodiments, many further alternatives, modifications, variations and applications 
will be apparent to those skilled in the art that in light of the foregoing description. 
Thus, the invention described herein is intended to embrace all such alternatives, 
25 modifications, variations and applications as may fall within the spirit and scope of 
the appended claims. 
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